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ABSTRACT 


The application of the product limit estimate (PL esti¬ 
mate) to the analysis of sonar detection range data was exam¬ 
ined using an empirical approach. A sonar detection distri¬ 
bution was assumed to be known and target lateral range dis¬ 
tributions corresponding to likely operational scenarios 
were derived. These distributions were used in a computer 
simulation to generate sample fleet exercise reconstruction 
data. For data of small sample sizes, the PL estimates of 
the sonar detection distribution were compared with the known 
distribution and were foxind to be reasonably accxarate approxi¬ 
mations for all tactical situations tested. . The PL estimates 
clearly outperformed the more readily computable, yet always 
optimistic, reduced sample (or empirical survival function) 
estimates. Finally, confidence intervals about the estimate 
were computed by using the jackknife procedure and the appar¬ 
ent optimism of the resulting confidence intervals was noted. 
Confidence interval baseline data is provided for future 
comparison with results from estimates generated from small 
sample sizes. Further research is indicated to improve the 
usefulness of the jackknife to this application. 
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I. INTRODUCTION 


Critical to antisxibmarine warfare (ASW) tactics and 
planning and to ASW analyses of all kinds are valid predic¬ 
tors of sonar detection performance. Meastires of perfor¬ 
mance, such as a sonar's detection range probability distri¬ 
bution, are important considerations to the at-sea commander 
in choosing proper force deployment for a particular ASW 
situation. In addition, these performance measures serve as 
inputs to computer simulations and wargames used for tactics 
development, specifying and testing requirements for new 
systems, and for training. 

Typically, analytically derived probability of detection 
models are based on the assumption of a detection occurring 
when the signal excess at the receiver (transducer) exceeds 
a given threshold value. Hence, probability of detection is 
equivalent to the probability of the signal excess exceeding 
the threshold. For a sonar system, signal excess is deter¬ 
mined by the sonar equation to be a function of the sonar's 
figure of merit (a measure of the system's characteristics) 
and signal transmission loss. Signal excess is usually 
assximed to have a known component and an unknown component 
taken to be a random variable with some ass\amed underlying 
distribution. In practice, these models are overly optimis¬ 
tic because they depend upon simplifying assiomptions which 
understate the complexities, for example, in the transmission 


8 


of sound through a medium such as the ocean. Recent examples 
are the probability of detection distributions for active 
sonars generated from models developed by Henry [1] which 
were shown to be optimistic estimates of actual operational 
sonar performance by Kelly [2]. 

It is intuitively appealing to use operational data to 
derive statistical estimates of a given sonar's probability 
of detection distribution; the estimate reflects actual 
sonar performance, not mathematical idealizations of it. 
Difficult conceptual problems are circumvented, such as how 
to deal with ping-to-ping correlation or how to model opera¬ 
tor response. Finally, using operational data to estimate 
the detection distribution results in a useful tool for model 
validation. Kelly has shown that the product limit estimate 
(PL estimate) can be used to estimate the distribution of 
sonar detection ranges from fleet exercise data [2] and an 
estimate of the same foimi has been recommended for use in 
Navy operational ASW analyses [3,4]. However, as Kelly 
points out, methods of computing confidence intervals about 
the estimate and methods for using the estimate in hypothe¬ 
sis testing have not been derived [2,5]:. Some of the PL 
estimate's intuitive appeal has, therefore, been lost because 
potential users have no procedure for determining the quality 
of the resulting estimate. 
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II. THE PRODUCT LIMIT ESTIMATE 


A. THE KAPLAN-MEIER ESTIMATE 

In 1958, Kaplan and Meier derived the product limit 
estimate, a nonparametric (distribution-free) maximum like¬ 
lihood estimate of lifelength distributions in which censor¬ 
ing of some test values may have occurred [6]. The derivation 
of the estimate assiimed that the lifelength and censoring 
distributions were independent. 

Censoring is said to result when some items of the test 
sample are either lost to the experiment, or the experiment 
ends before the items fail. In each case, time of item 
failure is unknown, or censored, either by item loss or 
experiment completion. The PL estimate makes use of infor¬ 
mation contained in the censored event; although failure 
time is not known, time of failiare must exceed the time 
from the moment at which the item was put on test until 
either it was lost to the experiment or the experiment ended. 

B. APPLICATION TO DETECTION RANGE DISTRIBUTIONS 

Similar censoring occurs in the analysis of detection 

range distributions. Consider a submarine closing on a 
surface ASW unit, for example a destroyer with active sonar. 
Let R, a random variable, be the detection range of the 
destroyer for this encounter. Let a random variable X be 
the distance to the submarine’s closest point of approach 
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(CPA), or lateral range, to the destroyer. If a definite 
range law is ass\imed to hold, then if R > X, a detection 
occurs. But, if X > R, a detection does not occur and the 
true value of R is censored by X. This censoring of detec¬ 
tion ranges by the lateral range is the basis for the 
applicability of the PL estimate. 

C. DEFINING THE APPLIED PL ESTIMATE 

Kelly has derived a PL estimate, denoted G(r), of the 
cumulative probability of detection distribution or P[R>r] 
and has shown that it has the same properties as (and is 
similar in form to) the Kaplan-Meier estimate. The proce¬ 
dure for computing the estimate for n encounters or data 
points is as follows: 

1. For each encounter i, record values of R^ or X^. 
X^ is usually available from exercise reconstruction data. 

2. Form the Z statistic such that 

= max(R^, Xj_) (1) 

3. Order the Z statistics such that 
2(1) < 2(2) < 

ine for all i = 1, ..., n the random variable 
1 if is a detection range 

( 2 ) 

0 if Z^^j is a lateral range only 
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5. Compute the PL estimate G(r) pointwise for 
3c *** if * * * / u 




G 


(k) 


°(k4-l) 

k+1 


(1 


°(k-H) , p 
k+1 ‘ (k+1) 


/V 



k = 1, ..., n-1 


(3) 


Sample data is shown in Table I with the associated 
estimates graphed in Figure 1. Also shown is the empirical 
survival function estimate, 1 - F(x), which Kaplan and Meier 
call the reduced sample estimate [6]. This estimate is 
computed from detection event data only and is defined by 
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''(k) - ^ ^ ''(k+1) 

(4) 


0 
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" "(n) 



This estimate is included for comparison with the PL estimate 
and because it has properties which allow some tests of 
statistical inference to be performed (such as the Kolmogorov 
Smirnov test) which have not been proven to be applicable 
to the PL estimate with censored samples. Further, it should 
be noted that the PL estimate from a sample space of all 
uncensored data yields the same result as the reduced 
estimate. 
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R(I) 


X( I } 


Z( 1} 


D{ I) 


9.04229 6.14829 9.04229 1 

13.60812 1.74829 13.60812 1 

20.93770 14.39638 20.93770 1 

10.01187 9.38966 10.01187 1 

12.79815 9.53916 12.79815 1 

14.20347 0.31195 14.20347 1 

8.46316 2.63672 8.46316 1 

19.70718 12.89940 19.70718 1 

20.00397 5.65837 20.00397 1 

14.57738 8.01824 14.57738 1 

10.17032 4.91411 10.17032 1 

7.74591 3.63399 7.74591 1 


Z( u 

D(I) 

GHAT(I) 

RS(I) 

4.87295 

0 

0.89513 

1.00000 

7.74591- 

1 

0.79027 

0.91667 

8.46316 

1 

0.68540 

0.83333 

9.04229 

1 

0.58053 

0.75000 

10.01187 

1 

0.47567 

0.66667 

10.17032 

1 

0.37080 

0.58333 

10.53208 

0 

0.37080 

0.58333 

11.92003 

0 

0.37080 

0.58333 

12.16581 

0 

0.37080 

0.58333 

12.79815 

1 

0.30089 

0.50000 

12.83923 

0 

0.30089 

0.50000 

13.60812 

1 

. 0.23733 

0.41667 

14.17439 

0 

0.23733 

0.41667 

14.20347 

1 

0.17867 

0.33333 

14.57738 

1 

0.12000 

0.25000 

16.48331 

0 

0.12000 

0.25000 

16.73190 

0 

0.12000 

0.25000 

16.92357 

0 

0.12000 

0.25000 

17.12126 

0 

0,12000 

0.25000 

18.40530 

0 

0.12000 

0.25000 

18.589 86 

0 

0.12000 

0.25000 

19.17041 

0 

0.12000 

0.25000 

19.70718 

1 

0.08000 

0.16667 

20.00397 

1 

0.04000 

0.08333 

20.93770 

1 

0.0 

0. 0 


TABLE I 
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0.0 6.25 12.50 18,75 25.00 


X-SCALE: 0.313E CO UNITS 

Y-SCALEi 0.167E-01 UNITS 


ESTIKATE PLOT (X*FFC;DUCT LIMIT ESTIMATE, » = REOUCeo SAMPLE ESTIMATEI 


1.00 


0.83 


0.67 


0.50 


0.33 


0.17 


0.0 


Figure 1 



Censored sample values have the effect of depressing 
the PL estimate from the uncensored or reduced sample 
estimate. It is this degradation of the estimate for missed 
detections which makes the PL estimate seem reasonable. 
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III. TESTING THE PRODUCT LIMIT ESTIMATE 


A. PROBLEM 

Application of the PL estimate to detection data has 
been limited in part because statistical information 
concerning the quality of the estimate is lacking. Actual 
sonar detection distributions are unknown and are therefore 
unavailable for comparison with the estimates of the 
distributions derived from operational data. The effects 
of differing lateral range censoring distributions are also 
not known; if certain lateral range distributions perturb 
the estimates greatly, then exercise data analysis ought to 
include estimates of the distributions of the lateral ranges. 
Further, although the PL estimate is a maxim\im likelihood 
estimate (MLE), the sample size of operational data is 
typically small so that large-sample properties of the MLE^ 
are not necessarily to be relied upon. The problem, then, 
is to investigate the behavior of the PL estimate compared 
to a known detection distribution when the sample size is 
realistically small. 

B. PROCEDURE 

The overall procedxire used was to assume a known 
probability of detection distribution for an arbitrary 
sonar system. Several lateral range distributions were 
derived based on plausible tactical situations. Encounters 
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between the submarine and destroyer were simulated on a 
computer; each encounter was called an event, or exercise 
event. A number of events constituted an exercise, the 
data from the exercise being used to construct the PL 
estimate. The estimate was then compared with the known 
detection distribution. 

1. The Assumed Distribution 

Based on the sonar equation and an assumed log 
normal distribution of signal excess, Henry [1] developed a 
family of detection distributions for various sonars 
operating under several "average” environmental conditions. 
The detection distribution assiimed to be known for the 
purposes of the simulation is shown in Figure 2 with ranges 
in kiloyards (kyd) and has the characteristic shape of 
Henry's distributions. 

Since the distribution could not be explicitly 
modeled or approximated by a well-known distributional 
form such as the Weibull or normal, twenty-six points were 
used to approximate the curve and were the inputs to the 
simulation. The probability integral transform method was 
used to sample for each event. 

2. The Lateral Range Distributions 

Lateral range distributions and sampling procedvires 
are given below. Sampling yielded a value of for each 
event to be paired with corresponding R^'s. 
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a. Random Search 

In this model/ the lateral range distribution 
is assumed to be uniform over the entire feasible range of 
the sonar, 0-20 kyd. This assumption is common throughout 
detection theory in describing a random search model. 

Therefore, the distribution function of X, is 

0 X < 0 

Fjj(x) = < ^ 0 < X < 20 (5) 

1 X > 20 . 

\ 

Sampling is achieved by using the probability 
integral transform method on the closed-form distribution 
function such that = 20 (U^) where U is uniformly distributed 
over the interval [0,1]. 

b. Barrier Search 

Consider the destroyer to be searching 
along a barrier front of width W, the barrier perpendicular 
to the submarine's estimated track, as in Figure 3 below. 


Submarine 

Track 


I 

I-•-1-[ 

0 V u| W 

Destroyer I 

Figure 3 
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X is then a two-dimensional random variable with distribution 
function F^(x) . Asstiming U and V to be uniformly distributed 

A 

over [0,W], then 


F„(x) = / P [v-x £ U £ v+x] f„(v)dv 
0 

X v+x W-x v+x 

= f j fy(u)f^(v)dudv + / / f^(u)f^(v)dudv 

00 X v-x 

w w 

+ J f f„(u)f (V) dudv 
W-x v-x 


X v+x 

= / 

0 0 


-ir dudv 

w 


w-x 



+ 


w 


/ 

w-x 


v+x , 

/ —J dudv 
v-x W 


/ —y dudv 

v-x W 


= 1 


. w-x . 2 

^ W ' 


( 6 ) 


Three barriers were considered: W = 25 kyd, 

W = 50 kyd, and W = 100 kyd. Sampling was done again by 
the probability integral transform such that = W(1 - ) 

where U is uniformly distributed over the interval [0,1], 
c. Submarine Stationkeeping 

For a submarine closing to maintain station a 
distance S from the destroyer (to act in an escort role, to 
trail the destroyer or to close for an attack from distance S), 
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consider the lateral range distribution to be normal, 
with mean S and standard deviation of 1 kyd. 

Sampling could have been accomplished by using 
a pre-programmed subroutine, but for speed and to minimize 
memory requirements, a statistical procedure was used which 
summed twelve uniform [0,1] random variates to yield a 
Normal(6,l) variate, then adding S-6 to achieve a mean 
equal to S. Values of S = 5 kyd and S = 10 kyd were tested. 
Since higher values for the standard deviation would tend 
to cause the lateral range distribution to be less peaked 
or spiked, and most likely perturb the estimate less, only 
a standard deviation of 1 kyd was tested. 

C. SIMULATION AND DATA 

An event i was simulated by sampling once from the known 

distribution to yield and once from the lateral range 

th 

distribution to yield Thus, for the i event, the 

s\:ibmarine would close on the destroyer to a range of kyd, 
while the destroyer's detection range for the event would 
be kyd. Twenty-five events were taken to be a realistic 
size for an exercise and the PL estimate and reduced sample 
estimate of detection range for each exercise were computed. 
Figures 4, 6 and 8 are the results of three exercises using 
the lateral range distribution associated with the random 
search model. Figures 5, 7, and 9 are percent or p-p plots 
[7] from the same respective cases; if the estimates and the 
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known distribution were identically distributed, the p-p 
plots would appear as straight lines. A reference line 
is plotted for comparison. 

Each pairing of known and lateral range distribution was 
used to simulate 100 exercises of 25 events each. Table II 
sijramarizes the results of the simulations for the six 
lateral range distributions. Averaged over the 100 
exercises are the PL estimates and reduced sample estimates 
at the 90^, 75^, 50^, 25^ and lo'’* percentiles. 

Fig\ire 10 plots the average residuals between the estimates 
and the known distribution. Details of the computer 
simulation may be found in the Appendix. 

D. CONCLUSIONS 

The simulation results show that for exercises of 
twenty-five events each, both the PL estimate and reduced 
sample estimate conform rather well to the known detection 
distribution; this is particularly true for the percentiles 
at middle and long ranges. More importantly, the PL estimate 
does appreciably better on the average than the reduced 
sample estimate. Examination of all plots similar to 
Figures 4, 6 and 8 for all exercises (600 total) has shown 
that, as expected, the reduced sample estimate is always 
more optimistic than the PL estimate; on the average, this 
is shown in Table II and Figure 10 as well. 

For exercises corresponding to the tactical situations 
of random search, barrier search for barriers of widths 
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I 


LATERAL 

RANGE 

DISTRIBUnCW 

ESTIMATE 

90 (6.15 kyd) 

PERCENTILES 

75 (8.70 kyd) 

(Known Distribution Value) 

50 (10.85 kyd) 25 (12.60 kyd) 

’ 

10 (15.05 kyd) 


Avg 

SD 

Avg 

SD 

Avg 

SD 

Avg 

SD 

Avg 

SD 

Uniform 

PL 

6.06 

2.34 

8.26 

1.77 

10.82 

1.00 

12.90 

1.09 

15.04 

1.49 

[0,20] 

RS 

8.24 

1.66 

9.96 

1.06 

11.91 

1.02 

13.96 

1.53 

16.32 

1.90 

Barrier 

PL 

5.94 

2.19 

8.53 

1.42 

10.84 

0.90 

12.82 

1.01 

15.06 

1.54 

(W = 25 kyd) 

RS 

7.68 

1.56 

9.74 

1.03 

11.67 

0.79 

13.63 

1.34 

16.06 

1.66 

Barrier 

PL 

6.27 

2.50 

8.26 

2.03 

10.71 

1.21 

12.74 

1.17 

14.95 

1.80 

(W = 50 kyd) 

RS 

7.90 

2.11 

9.86 

1.46 

11.80 

1.24 

13.75 

1.63 

16.15 

2.36 

Barrier 

PL 

7.49 

2.57 

8.28 

2.52 

10.61 

1.81 

12.78 

1.83 

15.39 

2.77 

(W = 100 kyd) 

RS 

8.49 

2.51 

10.36 

1.79 

12.39 

1.86 

14.40 

2.59 

15.91 

2.84 

Normal 

PL 

6.06 

1.46 

8.64 

1.02 

10.84 

0.82 

12.67 

0.91 

15.13 

1.37 

(5,1) 

RS 

7.65 

0.92 

9.27 

0.74 

11.23 

0.73 

13.14 

1.02 

15.21 

1.41 

Normal 

PL 

8.66 

0.60 

8.93 

0.71 

10.75 

0.91 

12.66 

0.92 

15.13 

1.37 

(10,1) 

RS 

10.36 

0.66 

11.33 

0.56 

12.50 

0.80 

14.38 

1.17 

16.38 

1.81 


TABLE II 
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25 kyd and 50 kyd and submarine stationkeeping of 5 kyd, 
the PL estimate appears to work extremely well. The lateral 
range distributions for these situations are characterized 
by expected values within the sonar's maximum detection range 
and variances such that samples tend to be spread throughout 
the sample space. As the submarine's CPA's become concen¬ 
trated further from the destroyer, however, the predictive 
value of the PL estimate appears to decrease at the 90^ 
percentile. For the barrier search with barrier width of 
100 kyd and the stationkeeping at 10 kyd, lateral ranges 
tend to concentrate at greater distances and censoring 
(degradation of the PL estimate) occurs at longer ranges. 
Hence, the estimate seems to be somewhat optimistic at 
short range. 

If it were feasible to actually run 100 exercises to 
estimate a sonar's detection distribution, then the results 
in Table II show that the PL estimate would do very well; 
the standard deviations of the percentile estimates would 
yield acceptably narrow confidence intervals. It should be 
noted that the data points for each percentile plotted as 
approximately straight lines on normal probability paper so 
that the assxamption that the data is normally distributed 
is not a bad one. However, the problem of setting confidence 
intervals on data from one exercise only is the challenging 
problem. A solution would be most useful to the analyst. 
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IV. CONFIDENCE INTERVALS 

A. : PROBLEM 

The nature of sonar detection data acquisition precludes 
obtaining large sample sizes except in the rare cases for 
which large-scale aggregation may be appropriate. Because 
of the time and expense involved in conducting exercises 
at sea, sample sizes are typically small. Miller [8] has 
shown that jackknifing the PL estimate produces confidence 
intervals which are asymptotically valid.(i.e. cover the 
true parameters with stated frequency), yet the asymptotic 
properties may not be valid for small samples. The problem 
is to determine the properties of the jackknife when it is 
used for computing confidence intervals for the PL estimate 
of the detection range distribution from data from a single 
exercise. 

B. PROCEDURE 

Using one lateral range distribution as a sample case, 
a single exercise of twenty-five encounters was computer 
simulated. Data was jackknifed according to the procedures 
described by Mosteller and Tukey [9]. The resulting pseudo¬ 
values were used to generate nominal 80%, 95% and 99% 
confidence intervals about selected percentiles of the 
estimate. One thousand exercises were simulated, and the 
number of times the confidence intervals covered the 
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percentiles of the known distribution was recorded. A 
simulation was also run with a correction factor inserted 
to adjust for possible pseudo-value? correlation, and the 
resulting confidence intervals were noted. Finally, the 
percentage of confidence intervals which actually covered 
each parameter was compared with the nominal or stated 
coverage. 

1. Lateral Range Distribution 

Of the lateral range distributions derived in 
Section III, the uniform'distribution associated with the 
random search model is probably the one which is applicable 
to a majority of exercise situations. For this reason, the 
uniformly distributed lateral range was used to examine 
the confidence intervals from the jackknife. 

2. Parameters 

Selected percentiles of the detection range 
distribution were taken as the parameters to be estimated 
by the jackknife. This was done because the percentiles 
have operational appeal; it is typical to want to know that 
50% of the time, detection range will exceed 8 kyd (±2 kyd 
at 95% confidence), say, than to want to know that for a 
target at 8 kyd or greater, probability of detection is 
0.5 (i.l5 at the 95% confidence level). In a sense, the 
two are equivalent in that bo-th can be used to generate 
confidence bands about the estimate. 
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3. The Jackknife 

The jackknife procedure yielded, for each exercise, 
pseudo-values which were used to generate confidence inter¬ 
vals. The pseudo-values resulted from PL estimates of the 
data with some data points omitted. 

For example, consider an exercise with twenty-four 
encounters. Twenty-four pseudo-values for each percentile 
can be generated by computing the PL estimate twenty-four 
times with a sample size of twenty-three, leaving out one 
observed value each time. Similarly, pseudo-values may 
be generated from grouped data. As an example, eight pseudo¬ 
values can be generated by computing the estimate eight 
times, omitting three different observations each time. An 
example of the results of the latter case is shown in Table 
III. 

According to Tiokey, the pseudo-values can be assumed 

to be normally distributed with their mean estimating the 

2 

percentile and the variances estimated by the usual s sta¬ 
tistic. Confidence intervals are obtained by using Student's 
t in the usual manner; however, one degree of freedom is 
lost for every duplicate pseudo-value. See Gray and Schucany 
[10] for asymptotic verification. 

4. Intraclass Correlation 

Little is known about the correlation between pseudo¬ 
values, although intuitively, correlation ought to exist to 
some degree. Consider two pseudo-values from a sample of 
twenty-four where one observation is omitted from the PL 
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PSEUDO-VALUES 


90 

75 

50 

25 

10 

9.03026 

10.96828 

11.42198 

13.22559 

15.33658 

8.92673 

10.83189 

11.82939 

21.17265 

15.33658 

2.00990 

6.20160 

11.02980 

16.65758 

21.57747 

2.34384 

9.06931 

11.39334 

13.21452 

15.33658 

a.73779 

10.58301 

11.44626 

16.94177 

21.57747 

9.02261 

9.20656 

10.28065 

13.22063 

15.33658 

8.90831 

10.80763 

11.56508 

16.65758 

21.57747 

8.96167 

10.87790 

11.97289 

16.95706 

15.47432 



ESTIMATES 



7.24264 

9.81827 

11.36742 

16.00592 

17.69412 



S/SQRT(N) 



1.10635 

0.58327 

0.18531 

0.96921 

1.13704 

T 

STATISTIC 

/ (DEGREES 

OF FREEDOM) 

2.36500 

2.36500 

2.36500 

2.44700 

4.30300 

( 7) 

{ 7) 

( 7) 

( 6) 

( 2) 


95? CONFIDENCE INTERVALS 


4.62612 

8.43884 

10.92916 

13.63427 

12.80144 

9.85916 

11. 19770 

11.80569 

18.37756 

22.58679 


TABLE III 
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estimate each time. Each pseudo-value, then, has twenty- 
two common data points used in computing the estimate and 
positive correlation between them would seem reasonable. 

Gray and Schucany [10], following Walsh [11], show 
that for correlation p between pseudo-values which are 
assumed to be samples from a multivariate normal population, 
the correlation factor for Student’s t statistic would be 

1 

They suggest that an adequate model of pseudo-value correla¬ 
tion for n pseudo-values is P = 1/n. Thus, the correlation 

1/2 

factor becomes [(2n-l)/(n-1)] ' . 

C. SIMULATION 

One thousand exercises of twenty-five events were simu¬ 
lated. Data from each exercise was grouped in five different 
ways, pseudo-values of the selected percentiles were computed 
using the jackknife for each grouping, and confidence inter¬ 
vals were computed for each percentile estimate. For groupings 
requiring only twenty-four data points, the final observation 
of each exercise (not the last ordered observation) was 
omitted. The Gray and Schucany correlation correction was 
applied to the sample of twenty-five pseudo-values. Results 
were compared with the percentiles of the known distribution 
and are shown in Tables IV(a)- ;iV(c). Details of the computer 
simulation are included in the Appendix. 
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SAMPLE SIZE 


DATA GEDUPS FOR 
JACKKNIFE 

A. % Coverage 

B. Average C.I. Width 

C. Average d.f. 


25 Pseudo-values ^ 

(25 groupings of 
24 data points each; „ 
in oniit 1 data point 

each time) „ 


80% CONFIDENCE 

90 

Percentil 

75 50 

e 

25 

10 

47.4 

45.5 

37.6 

40.9 

49.3 


4. 


10.8 


25 Pseudo-values A 65.7 

(25 of 24; emitting - 

1 each time) B 5.9 

S Incl. p correction - 

C 10.8 


A 44 2 

8 Pseudo-values 
(8 of 21; emitting „ , „ 

^ 3 each time) 



56.7 50.3 



43.9 47.1 


3.0 


57.1 


4.2 


6.9 


46.5 


5 Pseudo-valvies A 75.2 65.8 60.7 63.4 

(5 of 20; emitting- 

JQ 5 each time) B 10.8 6.2 3.2 3.2 


64.0 


5.6 


4.5 


39.1 


4.4 



3.998 


3.99 3.99 


51.7 


4.8 


3.4 


4 Pseudo—values A 52.6 60.3 64.0 64.6 60.3 

(4 of 18; omitting - 

S 6 each time) B 4.5 4.4 3.2 3.3 


3 Pseudo-values 

(3 of 16; emitting „ 

8 each time) 


2.996 


59.0 . 


5.0 


1.998 


TABLE IV(a) 


2.998 2.994 2.97 


68.5 


4.8 


69.3 


3.5 




71.9 71.9 


5.6 



1.993 1.96 
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SIZE 


data groups for 

JACKKNIFE 

A. % Coverage 

B. Avg. C.I. Width 

C. Average d.f. 


25 Pseudo-valioes 


25 Pseudo-valxaes 
Incl. p correc±ion 


8 Pseudo-values 


5 Pseudo-values 


4 Pseudo-vali:ies 


3 Pseudo-valxies 


95% CONFIDENCE 


Percentile 


< 

90 

75 

50 

25 

loi 

A 

70.8 

68.8 

59.5 

66.0 

74.4 

B 

6.5 

6.2 

4.9 

HI 

wm 

C 

10.8 

10.3 

8.9 

6.9 

4.5 

A 

82.2 

79.5 

75.0 

78.9 

78.7 

B 

9.2 

8.4 

6.3 

Hi 

10.7 

C 

10.8 

10.4 

Hi 

6.9 

4.5 

A 

68.2 

68.8 

72.2 

73.6 

63.0 

B 

6.6 

6.5 

HI 

5.2 

7.8 

C 

6.8 

6.8 

6.7 

6.2 

4.6 

A 

95.5 

88.4 

86.5 

86.2 

80.6 

B 

18.2 

10.9 

5.8 

5.8 

9.0 

C 

3.998 

3.99 

3.99 

3.9 

3.4 

A 

78.8 

87.1 

87.8 

89.0 

85.1 

B 

8.3 

8.3 

6.2 

6.5 

10.3 

C 

2.996 

2.998 

2.994 

2.97 

2.8 

A 

86.5 

90,0 

91.2 

91.3 

89.6 

B 

10.4 

10.4 

HI 

8.4 

13.2 

C 

1.998 

1.999 

2.0 

1.993 

1.96 


TABLE IV(b) 
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SAMPLE SIZE 

data groups K)R 
JACEOCNIEE 

A. % Coverage 

B. Avg. C.I. Width 

C. Average d.f. 

99% CONFIDENCE 

90 

I 

75 

’ercenti 

50 

le 

25 

10 

in 

OJ 

A 

25 Pseudo-valvies B 

C 

80.5 

81.8 

77.9 

80.6 

84.3 

9.0 

8.9 

8.1 

8.9 

20.9 

10.8 

10.3 

8.9 

6.9 

4.5 

in 

CN 

A 

25 Psevdo-values 

B 

Incl. p correction 

C 

89.1 

88.8 

88.1 

91.0 

87.0 

12.5 

•11.7 

9.1 

10.8 

17.5 

10.8 

10.4 

9.1 

6.9 

4.5 


A 

8 Pseudo-values B 

C 

80.2 

87.7 

88.2 

88.5 

81.6 

9.5 

9.6 

HI 


12.7 

6.8 

6.8 

HI 

6.2 

4.6 

in 

A 

5 Pseudo-values 3 

C 

98.4 

96.7 

94.6 

94.1 

90.3 

21.0 

16.4 

9.5 

9.5 

15.8 

3.998 

3.99 

3.99 

3.9 

3.4 

'i* 

04 

A 

4 Pseudo-values 3 

C 

92.4 

96.2 

95.6 

96.8 

92.4 

14.0 

14.0 

n.i 

11.7 

19.6 

2.996 

2.998 

2.994 

2.97 

2.8 

Ol 

A 

3 Pseudo-values 3 

C 

96.7 

97.6 

96.5 

98.1 

96.3 

21.0 

20.1 

16.4 

18.2 

29.8 

1.998 



1.99 

1.96 


I TABLE IV(c) 


3 
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i 
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D. DISCUSSION AND CONCLUSIONS 

j In every case, the confidence intervals computed from 
the jackknifed pseudovalues did not cover the known percen¬ 
tile value at the percentage specified, the nominal level. 

More importantly, the worst under-coverage occurred where 
confidence intervals were sufficiently narrow to be of real 

s 

! 

value; better coverage occurred only at the expense of wider 

confidence intervals. In analyzing 200 exercises in which 

i 

idle 95% confidence intervals did not capture the true percen- 

1 

I 

tiles in the five-group (twenty events each) runs, an approxi- 

I 

mately equal number of misses above and below the actual 

I 

values were found; therefore, no bias was indicated. Plots 
of groups of twenty-five pseudo-values on normal probability 
paper resulted in rather poor straight-line approximations 
indicating some doxibt in the assumption that these pseudo- 

I 

values could be normally distributed. Robust procedures, 
chiefly Winsorizing the pseudo-values before computing the 
confidence intervals, failed; the estimate (the average of 
the pseudo-values) depended heavily on pseudo-value outliers 
and accuracy decreased markedly where the outliers were 
Winsorized. 

! Use of the correlation correction improved confidence 
interval coverage, but in this case, was not as adequate a 
model as Gray and Schucany suggest. A value of p could be 
found by trial and error which would ensure proper coverage, 
but would require a determination of actual pseudo-value 
correlation for comparison and justification before it could 
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I 

be used. Also, it is likely that sufficiently wide confi¬ 
dence intervals would result that their usefulness would be 
degraded. 

i As discussed by Gaver [12], the tendency for the jack¬ 
knifed confidence intervals to cover the true parameter less 

frequently than specified is not uncommon. For small sample 

! 

sizes, when the jackknife may be the only tool available, 

I 

this possible tendency to under-cover must be acknowledged. 

J 

Resulting confidence intervals should be viewed as likely 

i 

i 

to be optimistic, although the correlation correction factor 

j 

is of help in improving the coverage. 
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APPENDIX A 


i 


j COMPUTER SIMULATION FOR TESTING EFFECTS 

I OF VARYING TACTICAL SITUATION 

The computer simulation is a discrete event digital simu- 

I 

1 

lation; each exercise event is a simulation event. The pro- 

I 

gram is written in the FORTRAN IV language, and is grouped 

i 

into easily identified structural sections shown in the 

I 

listing on page 45. 

) 

I 

I The following codes appear in the left margin to assist 

1 

potential users. Except for the C for "comment", these 

! 

codes are for documentation purposes only and must not 


appear in a program to be executed. 


CODE- 

C 

D 

OUT 


MEANING 

Comments 

Distribution; These cards determine 
the lateral range distribution to 
be sampled 

Output; These statements control 
the frequency at which interim re¬ 
sults are printed. As written, 
data from every 20th exercise is 
printed 


I One-dimensional arrays are listed in the order in which 
they appear and are used for the following purposes; 


ARRAY NAMES 


PURPOSE 


XS, YS 


Scaling arrays for the XY plot 
subroutine - contain axis endpoints 
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PPSUM, PESUM 


PPSSQ, PESSQ 

TILE 

XG 

PK, RK 

Z 

D, ZHOLD 

DELTA 
GHAT, ECDF 

DSUM 

PCDF, RCDF 

PTILE, EDTILE 

P90K 
a b 


Contain the stams of each of the five 
percentiles (90, 75, 50, 25, 10) for 
all exercises. PPSUM for the PL 
estimate and PESUM for the reduced 
sample estimate. Used to find aver¬ 
age percentile values at end of 
program. 

As for PPSUM and PESUM above, except 
contain the Siam of the squares of 
the percentiles. 

Contains percentile values which 
may be of interest (5% - 95% in 
increments of 5%) . 

Holds values to plot the 45® 
reference line.in the p-p plots. 

PK contains the probabilities of 
the known detection distribution 
associated with each range RK. 

Contains Z^'s (later, Z,.v's) for 
each exercise. 

Holds D, and Z. (unordered) values 
for each exercise. 

The array. 

Contains values of PL estimate (GHAT) 
reduced sample estimate (ECDF) corres¬ 
ponding to ranges in Z array. 

Accumulates the number of detection 
events for each exercise. 

Contains probabilities (PCDF) and 
corresponding ranges (RCDF) of 
reduced sample estimate. 

Hold percentiles of PL estimate 
(PTILE) and reduced sample estimate 
(EDTILE) corresponding to respective 
elements of TILE. 

Hold selected values of KTILE, 

PTILE, EDTILE: 

Part(a) indicates 90, 75, 50, 25, 
or 10th percentile 

Part(b) indicates KTILE, PTILE 
or EDTILE. 
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. 

1 

I 

I 

s 

i 

i , 

j AP, AE Contains average percentile values 

I for PL and reduced sample estimates 

I respectively. 

i 

i 

SP, SE Contains standard deviation of 

percentile values. 

1 

! 

I Subroutine RANDOM (ISEED/ U, 1) uses ISEED as the seed 

i 

f 

value for a random number generator and returns one ntimber 

1 

U which is uniformly distributed over the interval [0,1]. 

i 

Subroutine OVFLOW suppresses overflow errors caused by RANDOM, 
j Program output is in two forms: 

I 1. Interim Output 

I For each selected exercise, output consists of 

a listing of the 's, 's and the PL and reduced esti¬ 

mates as in the lower half of Table I, a plot of the assumed 

distribution and the estimates as in Figure 4, and a p-p 

{ 

plot of the same form as Figure 5. 

I 2. Final Output 

A table is generated showing the five percentile 
(90, 75, 50, 25 and 10) values generated by the estimates 

for each of 100 exercises. The averages and standard devia- 

[ 

tions for each percentile over the 100 exercises are also 

{ 

printed. 
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aa 


CdLL QVFLOW 
ISEED=490214 

INTEGER 0(25y,DELTA(25>,OSUM 

REAL RK(26)tPK(26),Z(25),ZH0L0(25),GHAT(25),XS(2)t 
1PCDF{25)fRCDF(25),XG(12),TILE(19),KTILE(19) , PTILE{1 
2P90K(100 )tP 90P(100),P90E(100),P75K(100),P75P(100), 
fpSOKdOOl,P50P(100),P50£(100),P25K(100)»P25P(100), 
4P1CK(100),P10P<100)iPlOEl100),PPSUM(5),PESUM(5), 
5AP(5),AE(5)tSP(5),St(5)»ECDF(25)»YSI 2),EDTILE(19), 
6P75E(100),P25E(100),PPSSQ(5),PESSQ(5) 

XS(1) = 0-0 
XS(2) = 25.0001 
YS(1) = 0*0 
YS(2J - I. 0001 
DC 5 I - 1,12^ 

XG(I) = 2. I 
5 CONTINUE 

DC 10 I = lt5 
PFSUMd) - 0,0 
PESUMtI) = 0.0 
PPSSQd ) 5= 0.0 
FESSQU ) * 0.0 
10 CONTINUE 

SET PERCENTILES REQUIRED 

DC 15 I = 1,19 
TILEd) = I .05 
15 CONTINUE 

READ IN KNOWN PROBABILITY OF DETECTION DISTRIBUTION 

READ! 5,2001 {PK( I ) ,RKd) ,1 = 1,26) 

200 FCRMATI2F10.5) 

DC 95 M = 1,100 

GENERATE RANDOM VARIATES 


DC 25 I = 1,25 

CALL RANDOM!ISEE0,U,1) 


X = 20. * U 

CALL RANDOM!ISEED,U,1) 

DO 20 J = 1,26 
IF(U.GT.PK(J)) GO TO 20 

P = RK(jr + (PK( J)-U)#(RK(J-l)-RK(J))/(PK(J)-FK(J-l)) 
GG TO 21 

20 CONTINUE 

21 IF(P,GE.X) GO TO 22 
Zd) = X 

Cd) = 0 
GO TO 25 

22 Zd) = R 
Dd) = 1 

25 CONTINUE 

COPY Z ARRAY INTO Z HOLD, D ARRAY INTO DELTA 

DC 30 I = 1,25 
ZHCLOd ) :? Zd) 

OELTAdl = Dd) 

ECDFd) = O.O 
30 CONTINUE 

SCRT Z ARRAY 

CALL PXS0RT(Z,1,25) 

MATCH DELTAS TO SORTED ARRAY 

DC 40 I = 1,25 
DC ^5 J = 1,25 

IF(Zd) .NE.ZHOLD! J) ) GO TO 35 
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DEtTA< J J = D{J) 

GC TO 40 
35 CONTINUE 
40 CONTINUE 

C I COMPUTE PRODUCT LIMIT ESTIMATE 

I GHATt25) =0.0 
DC 45 J = 1»24 
K = 26 J 

Xl“=^FLQAT(DELTA(K)J/FLOAT(K) 

GHAT(L) = XI <• (1. - XI)=«'GHAT(K) 

45 CONTINUE 

COMPUTE ECDF FROM DETECTION EVENTS ONLY 

DSUM =0 . 

DC 50 I = lf25 
RCDFtll =0.0 
PCDF(I) = 0.0 ^ . 

CSUM = CSUM DELTA( I ) 

CONTINUE 
K = 1 

DC 60 I = ItOSUM 
DC 55 J = K,25 
if<delta(j).eq.o) go to 51 
RCDF(I» = Z(J) 

- 1. - FLOATd )/FLOAT(DSUM) 


50 


51 

55 

60 


PCDF(I) = 
K = K ♦ 1 
GC TO 60 
K = K + l 
CONTINUE 
CONTINUE 


C| COMPUTE SELECTED PERCENTILES 

! DC 80 I = 1,19 
i X = TILEd ) 

I DC 65 K = 1,26 
J = 27 - K 

I IFIX.LT.PKIJ)} GO TO 65 
! IF(PK(J)-PK(J-1)>61,61,62 

61 KTILE(I» = 21.0 
GC TO 66 

62 A = PK( J)-FK( J-1 ) 

i KTILEdl=RK( J)+(RK(J^1)-RK( J) j’i'IPKIJl-X) / A 
GC TO 66 
: 65 CONTINUE 
i 66 DC 70 J = 1,25 

IF(X.GT.GHAT(J>) GO TO 71 
! 70 CONTINUE 
I 71 PTILE(I) = Z(J) 
i DC 75 J = 1,25 
j IF(X.GT.PCDF(J)) GO TO 76 
! 75 CONTINUE 
i 76 ECTILEII) = RCDF(J) 

I 80 CONTINUE 
P9CK(M) 


P75K(M) 

PSGKtMl 

P25K(M) 

PIOK(M) 

P9CP(M) 

P75P(M) 

P50P(M> 

P25P(M> 

FIOP(M) 

P90E(M) 

P75E(M) 

P5CE(M) 

P25E(M) 


KTILE(18) 
KTILE(15) 
KTILE(IO) 
KTILE(5) 
KTILEI2) 
PTILE(18) 
PTILEI15) 
PTILEI10) 
PTILE(5) 
PTILEI2) 
EDTILE(18) 
EDTILEI15) 
EDTiLEdO) 
EDTILE(5) 
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F10E(M} = 
PPSUM(l) 
PPSUM(2) 
PFSU^O » 
PFSUM(4) 
PPSUM(5) 
PESUM(1) 
PESUM(2) 
FESUMO J 
PESUM(4) 
PESUM(5) 
FPSSQ(l) 
FFSSQ(2) 
PPSSQO) 
PPSSQ(4) 
PPSS^}(5) 
FESSQd J 
PESSQ(2) 
PESSQO) 
FESSQ(4J 
PESSQ(5) 


EDTILE(2 
PPSUM(1 
PPSUM{2 
PPSUMO, 
PPSUM(4 
PPSUM(5 
PESUM(1 
PESUM(2 
PESUMO 
PESUM(4 
PESUM(S 
PPSSQ( 1 
PPSSQ(2 
PPSSQO 
PPSSQ(4 
PPSSQtS 
PESSQ( 1 
PESSQ(2 
PESSQO 
PESSQ(4 
PESSQ(5 


P90P(M) 

P75P{M) 

P50P(M) 

P25P(M) 

PIOP(M) 

P90E(M) 

P75E(M) 

P50E(M) 

P25E(M) 

PIOECM) 

(P90P(M) 

<P75P(M> 

(P50P(M) 

(P25P(M) 

(PIOP(M) 

(P90E(M) 

(P75E{M) 

(P50E(M) 

(P25E(M) 

(PIOE(M) 


) 

) 

’ 4 '* 2 ) 

4 «=> 2 ) 


PRINT AND PLOT SELECTED INTERIM RESULTS 


ICHECK) 95Tai»95 


OUT ICHECK = M/20 

OUT IF(FL0AT(M)/20.- ICHECK) 95Tai,95 
ai HRITE(6,250) 

25C FCFMATtlHl) 

CALL PLGTP(XSfYSf2,l) 
call PLCTP(RK,PK,26,2) 

CALL fLCTP(RCDF,PCDF,25,2) 

CALL PLGTP{Z,GHAT,25,3) 

WRITE(6,300) 

300 FGFMATtlX//8Xt'DISTRIBUTION PLOT (+=ASSUMED DISTRIBUT 
IICN, X=FRGDUCT LIMIT ESTIMATE, ♦=RE0UC£D SAMPL-E ESTIMA 
2TE)'//) 

WRITE(6t325) 

325 FCPMAT(lHl) 

CALL FLCTP(XS,XS,2,U 
CALL PL0TP(XGtXG,12t2) 

CALL FLGTPIKTILE,EDTILE,19,2) 

CALL FLGTPIKTILE,PTILE,19,3) 

WRITE{6,350) 

350 FORMAT!1X//8X,'P~P PLOT (♦=REFERENCE, X=PRCDUCT LIMIT 
1 ESTIMATE, *=REDUCED SAMPLE ESTIMATE)'//) 

400 FORMAT!1H1/,19X,'Z(I)',4X,•DELTA!I)',3X,'GHAT(I)•,6X, 
l'FS!I)'//) 

N = 1 

DC 85 I = 1,25 
IFIDELTA! D.NE.O) GO TO 86 
ECCF!I) = 1.0 

85 CONTINUE 

86 DC 90 K = 1,25 
IF!DELTA(K).EQ.O) GO TO 91 
ECDF!K) = PCDF!N) 

N = N 1 
GC TO 90 

91 L = N - 1 

ECDF!K) = PCDF!L) 

90 CONTINUE 

400 FOFMAT!1H0?18X,'Z(I)',4X,'D!I)',3X,'GHAT(I)',6X, 

^WRITE!6(450)(Z!I),DELTA!I),GHAT!1),eCDF!I),1=1,25) 

450 FORMAT!15X,F10.5,4X,II,4X,F7.5,2X,F10.5) 

95 CONTINUE 

PRINT FINAL RESULTS 

500 FORMAT!iHl?20X,'P90',21X,'P75*,21X,'P50’,21X,'P25', 
121X,'PIO'/) 


PCDF!L) 
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DC 105 I = 1,100 

1 ’ jP90E(I) ,P75K<I ) ,P75P( IJ 

: 8? ;?!§! 

105 CONTINUE 

550 ^°f^MAT(13X,5(|(F5.2,2X),3X)) 

AF(IJ = PPSUM(I) / 100. 

AE(I) = PESUM(I) / 100. 

ccitI ^ I “ (PPSUM(I)*’!'2)/100. )/99. ) 

,nn “ ( PESUM( I)*’!'2) / 1 00 . )/99. } 

100 CONTINUE 

WRITE(6|600) (AP(I),AE(I) ,I = 1,5) 

600 FOPMATIix//l4X,•AVG*,3X,4(F5.2,ix,F5.2, 

L P 3 • Z i 

HPITE(6,650)(SP(J),SE{J),J=1,5) 

650 FORMAT!14X,*S.D.•,2X,4(F5.2,2X,F5.2,12X),F5.2,2X, 

1 F5.2 ) 

STOP 
END 


12X),F5.2»2X, 


DATA 

KNOWN DISTRIBUTION 


0.00 

21.0 

0.04 

17.0 

0.08 

15.7 

0.12 

14.8 

0.16 

14.0 

0.20 

13.1 

0.24 

12.7 

0.28 . 

12.4 

0.32 

12.1 

0.36 

11.9 

0.40 

11.6 

0.44 

11.3 

0.48 

11.0 

0.52 

10.6 

0.56 

10.3 

0.60 

10.0 

0.64 

9.8 

0.68 

9.4 

0.72 

9.0 

0.76 

8.6 

0.8 

8.0 

0.84 

7.5 

0.88 

6.6 

0.92 

5.0 

0.96 

2.5 

1.00 

0.0 
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appendix b 


COMPUTER SIMULATION WITH JACKKNIFED 
CONFIDENCE INTERVALS 

The structure of this simulation is similar to the one 
described in Appendix A. Differences are discussed below. 
Margin codes: 


CODE 

C 

N 


0 - 


I 


NEX 


MEANING 

Comments 

These statements control the 
number of pseudo-values which 
result 

These cards control the number 
of observations omitted from 
each jackknifed estimate 

Statements which control which 
observations are included in 
each jackknife 

Controls the number of exercises 
to be simulated 


COR Statements to which correlation 

correction factors can be added 


ARRAY NAMES PURPOSE 

FREQ Acctimulates the number of times 

the confidence intervals cover 
each known percentile 

AVDF Contains the average number of 

degrees of freedom for each 
percentile estimate 

CDIST Contains the average confidence 

interval width for each percentile 
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ZSM, GSM 


Zl, D1 


THETA 
THEO, THEI 


SUM, SQSUM 
CL, CU 

DF 


Contains values of the PL estimate 
(GSM) at corresponding ranges 
(ZSM) after the estimate has been 
smoothed 

Contains values of Z,.. and D ... 
after observations hivfe been 
omitted for the jackknife 

The pseudo-values 

The values used in computing the 
pseudovalue: THEO contains the 
percentile value when all Z...'s 
are used in the estimate, 

THEI the value when observations 
are omitted. 

Accumulates the sums and sums of 
squares of the pseudo-values 

Lower and upper confidence 
intervals for each percentile 

Contains the degrees of freedom 
used to estimate each percentile 
for each exercise 


For each exercise, output includes a table similar to 
Table III. After all exercises have been simulated, confi¬ 
dence interval coverage frequency, average confidence inter¬ 
val widths, and average degree of freedom used to determine 
the t statistic for each percentile are listed. For large 
numbers of exercises, printouts of the pseudo-value tables 
can be suppressed. 
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CALL CVFLQA^ 

I SEED = 530408 

NDl integer D<25)tDELTA(25)♦DF(5),FREQ(5) 

RK(26)»PK(26),zT 25|.ZH0LD(25) tGHAT ( 25 ) *1ILE ( 5 )» 
N7 1 tSUM^),ZSM(52),GSM(52). 

N2l 3^7jLE(i) JcDISTcii t Zl( 21) ,AVDF(5) t 

C AKDMRCENTILErOF''?i^I^ST'^ DETECTION DISTRIBUTION 

200 FCFMAKFIO.S) 

OG 5 I = 1,5 
FREQ(I) = 0 
AVOFd) = 0.0 
CDISTd) = 0.0 
5 CCNTINUE 

NEX DO 175 N1 = 1,5 

C GENERATE RANDOM VAR IAT5S 

DO 15 I = 1,25 

CALL RANDOM( ISEED,U,1) 

X = 20. « U 

CALL RANDaM(ISEEO,U,l) 

DC 10 J = 1,26 
IF(U.GT.PK(J)) GO TO 10 

R = RK(J) + (PK( J)-U)=«'(RK{J-1)”RK(J))/(PK(J)-PK(J-1) ) 
GO TO 11 

10 CONTINUE 

11 IF(R.GE.X) GO TO 12 
Z(I) - X 

0(1) = 0 
GO TO 15 

12 Z(I) s R 
0(1) * I 

15 CCNTINUE 

C COPY Z ARRAY INTO Z HOLD, D ARRAY INTO DELTA 

DC 20 I = 1,25 
ZHCLD(I) = Z(I) 

DELTA!I) - D(I) 

20 CONTINUE 

C SORT Z ARRAY 

CALL PXS0RT(Z,1,25) 

C MATCH DELTAS TO SORTED ARRAY 

DO 30 I = 1,25 
DO 25 J = 1,25 

IF(Z(I) .NE.ZHOLO!J)) GO TO 25 
DELTA(I) = D(J) 

GO TO 30 
25 CCNTINUE 
3C CCNTINUE 

C COMPUTE PRODUCT LIMIT ESTIMATE FOR SELECTED PERCENTILES 

GHAT(25) = 0.0 
DC 35 J = 1,24 
K = 26 - J 
L = 25 - J 

XI = FLQAT(DELTA(K))/FLOAT(K) , , 

GHAT(L> = XI + (1. - XI)*GHAT(K) 

35 CCNTINUE 
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SMOOTHING THE ESTIMATE 


40 


ZSM(l) 
GSM(l) 
ZSM(2) 
GSM(2) 
ZSM(3) 
GSM<3) 

DC 40 I 
J1 = 

J2 = J1 
J = I <- 
ZSM(Jl) 
GSM(^l) = 
ZSM(J2) = 
GSM(J2) = 
CONTINUE 
ZSM(52) = 
GSM(52} = 


( 2 * 1)+2 
^ I 
1 


0.0 

1.0 

Z(l)/2. 

1.0 

Z(l) 

(1. GHAT{L))/2. 

It24 


(Z{I)+Z(J))/2. 

GHAT(I) 

Z( J) 

(GHAT{I)+GHAT(J)1/2. 


Z(25J 

0.0 


+ (ZSM(51)-ZSM(50)) 


45 

50 


FIND SELECTED PERCENTILES 

DC 50 J = 1,5 
DC 45 K = 1,52 

IF(GSM(K).GT.TILE(J)) GO TO 45 

A = (TILE( J)-GSM(KU*{ZSM(K)-ZSM(K-1)) 

THEO(J) = ZSM(K) - A/(GSM(K-1)-GSM(K)) 

GO TO 50 

CONTINUE 

CONTINUE 


C 
C 

N 

C,0 

0 

0 

I 


CCMPLTE PRODUCT LIMIT ESTIMATE FOR SELECTED PERCENTILES 
WITH 'INCOMPLETE' DATA 


DO 95 M = 1,8 
OMIT 3 ELEMENTS 


PER RUN FROM DATA SET 


L = (3*M)+1 
K = L-3 

DC 55 I = 1,21 
IF(I.EQ.K) GO TO 56 

ZKI) = ZHOLD(I) 

55 CONTINUE 

I 56 DO 60 J = 1,21 
I IF(J.GT.21) GO TO 61 

ZKJ) = ZHCLD(L) 

L L+1 

60 CCNTINUE 

C SCRT Z1 ARRAY AND MATCH DELTAS 

I 61 CALL PXSORT(Zl,l,21) 

I DC 70 I = 1,21 

DO 65 J = 1,25 

IF(Z1(I).NE.ZH0L0(J)) GO TO 65 
OKI) = D(J) 

65 CCNTINUE 
70 CONTINUE 

C COMPUTE THE PRODUCT LIMIT ESTIMATE ('INCOMPLETE' DATA) 

I GHAT(21) = 0.0 

I DC 75 J = 1,20 

I K = 22 - J 

^ Xl”=^FLGAT(Dl(K))/FLOAT(K) 

GHAT(L) = XI + (l.-XI)*GHAT(K) 

75 CCNTINUE 

C SMOOTHING THE ESTIMATE 
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ZSM(2) = Zl(l)/2. 

ZSM(3) = ZL(1) 

GSM(3) = (1. GHAT(l))/2. 

DO 80 I = It 20 
J1 = (2=*'I)<*2 
J2 = Jl+1 
J = I + ] 

ZSM(Jl) = 

GSM(Jl) ^ 

ZSM(J2} = 

GSM(J2) = 

80 CONTINUE 

hsH\tt\ = " (ZSm43,-ZSN(42)) 


(Zl(I)+Z1(J))/2. 
GHAT(I) 

Zl( J) 

(GHAT(I)+GHAr(J))/2. 


C FIND SELECTED PERCENTILES 

DC 90 J = Lt5 
I OC 85 K = 1,44 

IF(GSM(K).GT.TILE(J)) GOTO 85 
A = (TILE(J)-GSM(K))*(ZSM(K)-ZSM(K'n) 
THEIIJtM) = Z:SM(K) - A/(GSM(K-1)-GSM(K) ) 
GO TO 90 
85 CONTINUE 
90 CONTINUE 
95 CONTINUE 

C PERFORM JACKKNIFE 

DC 105 I = 1,5 
DF(U = 0 
SUM(I) =0.0 
SCSUMII) = 0.0 
105 CONTINUE 

DC 125 I =1,5 
N DC 115 J = 1,8 
N THETAd ,J) = 8.=4'THEO( I) 


- 7.*THEI(I,J) 


GATHER STATISTICS 


N 

N 


SUM(I) = SUMII) + THETAd,J) 

SCSUM(I) = SQSUM(I) + THETA! I, J)=«i#2 
115 CONTINUE 
125 CONTINUE 

COMPUTE DEGREES OF FREEDOM 

DC 155 I = 1,5 
DO 145 J = 1,8 
K = J<*1 

DC 135 L = K,8 

IFITHETAI I ,J) .EQ.THETAd,D) GO TO 145 
135 CONTINUE 

DFd) = DFd) * 1 
145 CONTINUE 

0F( I) = DFd) - 1 
155 CONTINUE 

C COMPUTE 95% CONFIDENCE INTERVALS 
DO 165 I = 1,5 

N GFATd) = SUM( I)/8. ^ 

N SSTATd) = SQRT( ( SQSUMd )-( SUMCI )=<'=<‘2)/3 . )/7.)/SQRT(8.) 

COR CLd) = GHATCI) - T(DF( I) )=<'SSTAT d ) 

IFICLd) .LT.O.O) CLd) =0.0 
COR cud) = GHAT(I) + T (D F d ) )’S'SST AT (I) 

AVDF( I) = AVDFd ) + DFd) 

CDIST(I) = COISTd) cud) - CLd) 

IF((CL{ I ).GT.KTILEd ) ).0R. (KTiLEd) .GT.CUI I )) )GQTO 165 
FRECd) = FREQd) + 1 
165 CONTINUE 
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C PRINT RESULTS 


WRITE(6,300) 

WRITE(6,400) 

FCPMAT(1H0»37X,‘ESTIMATES *} 
WRITE(6,450}(GHATin,1=1,5) 
FGRMATtlHO,l5X,5F10.5/> 
WRITE<6,500) 

FORMAT(1H0,37X,* S/SQRT(N)*) 
WRITE(6,450)(S$TAT(I),1=1,5) 
WRITE(6,550) 


300 

350 

400 

45C 

500 

550 

650 

700 

175 

850 

185 

900 

950 


, * 75 *, 


FORMAT!1H0,23X,'T STATISTIC 
WRITE(6,450){T(DF(I)) ,1 = 1,5) 
WRITE!6,650) 1 OF!I) , I = 1,5) 


/ !DEGREES OF FREEDOM)') 


INTERVALS'/) 


f.S??^Ti^ynAi5X»5!4X,'f',I2,')',2X)) 

WRITE!6,700) 

FORMAT!IHO♦29X,'95% CONFIDENCE 
WRITE!6,350)!CL!I),1=1,5) 

WRITE!6,350)!CU!I),1=1,5) 

CONTINUE 

WRITE!6,850)!FREQ!I),1=1,5) 

FORMAT!IHO,37X,'FREQUENCY'//,15X,5!4X,13,3X)) 
DC 185 I = 1,5 

CDISTII) = CDIST!I)/FL0AT!N1) 

AVDFII) = AVDF!I)/FLOAT!Nl) 

CONTINUE 

WRITE!6,900){CD I ST!I),1 = 1,5) 

FORMAT!IHO,32X,'AVERAGE C.T. 

WRITE!6,950)!AVDF!I),1=1,5) 

FORMAT!IHO,28X,'AVERAGE DEGREES 
15F10.5) 

STOP 
END 


WIDTH'//,15X,5F10.5) 


OF FREEDOM'//,15Xi 


DATA 

PERCENTILES OF INTEREST 


0.90 

6.15 

0.75 

8.7 

0.50 

10.85 

0.25 

12.6 

0.10 

15.05 


KNOWN CISTRI 

0.00 

21.0 

0.04 

17,0 

0.08 

15,7 

0.12 

14.8 

0.16 

14.0 

0.20 

13.1 

0.24 

12.7 

0.28 

12.4 

0.32 

12.1 

0.36 

11.9 

0.40 

11.6 

0.44 

11.3 

0.48 

11,0 

0.52 

10.6 

0.56 

10.3 

0.60 

10.0 

0.64 

9.8 

0.68 

9.4 

0.72 

9.0 
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0.76 

3.6 

0.8 

8.0 

0.84 

7.5 

0.88 

6.6 

0.92 

5.0 

0.96 

2.5 

1.00 

0.0 


VALUES OF T STATISTIC (95?f Z-SIDED TEST) 

12.7C6 

4.303 

3.182 

2.776 

2.571 

2.44? 

2.365 

2.306 

2.262 

2.228 

2.201 

2.17S 

2.160 

2.145 

2.131 

2.120 

2.110 

2.101 

2.093 

2.086 

2.080 

2.074 

2.069 

2.064 
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